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motor sensory functions from the neck and below, upper neck functions such 
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the movement of his/her eyeballs to convey desired information. Although 
many similar researches have been done, this paper proposes the use of 
image processing on image captured using webcam with its Infra-Red (IR) 
Pupil detection filter removed (a.k.a night vision) to achieve robustness. This allows the 
algorithm to properly track the location of the iris despite of its and the pupil 
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owns tradeoff between speed and accuracy. Analysis on both algorithms 
shows good tracking performance despite of the mentioned tradeoff 
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1, INTRODUCTION 

Our freedom to move freely is invaluable to us. However not everyone shared the same taste of 
freedom. For some, such as a quadriplegic patient, or a paralyzed person, unable to move independently is 
dementing. To overcome this matter many researches have been done in the medical field to improve the 
quality of life of a quadriplegic. The fact that most quadriplegic retain most of its upper neck function, give 
way for researcher to leverage this into assisting them to move independently again [1]-[3]. In most 
researches an electric wheelchair is used to achieve this objective [1], [4]. Bio-signals such as the movement 
of the eyeball [5], [4], EEG signals and EMG signals are translated in command to move the wheelchair [4]. 
This project proposes the use of image processing technique to track the movement of the eyeball, by 
locating subject’s pupil. The location can later be translated into commands to move the wheelchair to left or 
right. This leads the researchers to develop a system that can help the disable person by moving the eye 
balls. 

Generally, gaze detection system is a system that captured image data of eye pupil, detecting and 
tracking movements of the user’s eyes, approximating the line-of-sight vector, and move the electric 
wheelchair in desired direction based on the person’s eyes and it is a system based on eye biometrics [6]. Eye 
tracking system/gaze tracking system use the image processing technique to convert the input data into 
digital form and some mathematical operations are applied to the data to create more image enhancement [7]. 
Using image processing technique, the image of the picture will then be identified on colors, shades, 
contrasts and shapes that cannot be seen by normal eye. Subsequently, after performing image processing 
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technique, information on the location of the pupil will then be sent to a microprocessor that will control the 
directional movement of a wheelchair motor [16]. 

Edge detection is an image processing technique for finding the boundaries of objects within images 
[8]. This can be achieved by detecting discontinuities in brightness. Edge detection is castoff for image 
segmentation and data extraction in areas such as image processing, computer vision, and machine vision. 
Common edge detection algorithms include Sobel, Canny, Prewitt, Roberts, and fuzzy logic methods. But in 
this case, the use of canny method is essential as the input frame is processed for digital images [9]. This 
method is not finalized since it is in the middle of process. The result of edge detection method will be 
continued with some morphology fundamental for pupil extraction. 

This paper proposes two type of method in order to detect the iris. The first step is to detect the user 
face by using skin color segmentation and find the face of the person over the image. This method 
categorized the image into facial and non-facial region since human skin color is dissimilar from the nature. 
After the face of the person is detected, the proposed used of circular Hough Transform permitted to locate 
the iris in the face region. This project will focus more on eye detection. The Hough transform is a feature 
abstraction technique used in image analysis, computer visualization, and digital image processing [10]. The 
purpose of this transform is to discover imperfect occasions of object inside a certain class of shapes by a 
voting procedure. This method is considered to be suitable since the circular shapes can detect the pupil 
region [11], [12]. 

Regionprops is also used in this project, which it can used to calculate object properties from the 
binary image. Every regionprops has some specific properties that can be used to measure the binary image. 
Some of example properties that can be use is bounding box properties. This is represented as a 4-vector 
where the first two entries are the x and y coordinates of the upper left corner of the bounding box, and the 
two last entries are the width and the height of the box [13]. The suitable properties that can be use are 
centroid properties that calculate the center coordination of binary object. The centroid will be used gaze 
detection purposed. 


2. RESEARCH METHOD 

This system consists of 3 parts which is the hardware development for the first part. After the 
hardware configuration is ready, the next step will be the algorithm to track the pupil region. The second part 
will shows the process under taken to extract pupil boundary. The last part 1s to formulate the gaze tracking 
direction. 





Figure 1. Overall gaze detection system 


2.1. Headset Configuration 

This project consists of several devices that need to be considered for the accuracy of webcam 
placement. Therefore, a head set 1s design by using SOLIDWORK so that the webcam can be placed on top 
of the head to identify the gaze position. 
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Figure 2. Design of gaze detection headset 


2.2. Night Vision Camera 

The contour and color for both pupil and iris of Asian people is more to dark brown, unlike 
European people which its eyes intensity between pupil and iris can be seen by clear vision or normal colored 
webcam [14]. To overcome this problem, the use of night vision camera can be used to overcome the 
problems since it can clearly differentiate between pupil and iris of the human eyes. 

The use or real Night Vision Camera is expensive. However this can be achieve by removing the IR 
filter of a normal cheap webcam such as shown in Figure 3. By removing the IR filter, it allows the infrared 
light to be detected by the night vision camera. 

Normal LED tends to make the patient or user feels uncomfortable due to light intensity that 
immerse towards the eyes. The function of the IR LED is to provide an additional light towards the human 
vision so that it can be detected by night vision camera. The advantages of this camera are it detect the iris 
and provide additional light that cannot be seen by normal vision. 





Figure 3. Webcam with night vision 


2.3. Algorithm for Gaze Tracking 

For both algorithms, the process of Image Acquisition, Edge Detection, Image Filled and Creates a 
Structural Element are identical [15]. The different comes when the pupil is detected and it needs to be 
centralized. Both Algorithms will use different medium in order to identify the coordinates and to highlight 
the binary region. 
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Figure 4: Development of gaze tracking algorithm 


2.4. Image Acquisition 

First part of this project 1s by capturing the image which will be in an RGB image. The image 
acquisition will frame these images from recorded video and stored in the MATLAB directory. The image 
after that will be converted into a gray scale image since the pixel stored in the images is mostly bright and 
dark. After that, the gray scale image is converted into binary images that represent 0 for black and white for 
1. This will result in black and white images. As we know that our pupil is mostly black, thus it wills easily 
getting the region as we convert it into binary. 


2.5. Edge Detection 

After the binary image is obtained with desired threshold level, the next process would be detecting 
the edges of the area of interest in the image which is the pupil portion, in order to remove the portion of the 
pupil out from the enhanced image. The chosen filter which is the canny filter has been implemented to the 
images in order to get the noticeable edges. The canny edge detector first smooths the image to eliminate and 
reduce noise. It then finds the image gradient to highlight regions with high spatial derivatives which mean 
the sharp changes in image brightness to get important information from the image. Canny filter been chosen 
due to the characteristic of the eyes which is round and curved, thus it is can produced the desired result. 


2.6. Fill Holes 

After the final result we will get the desired edge which is the pupil center, it is an essential 
requirement to detect the movement of the human eyes. As we know, pupil portion has been used for tracking 
the gaze movement either right or left. In order to make the pupil portion to be more noticeable, dilation 
process has been applied towards the image. Then the morphological operation is implemented to the image. 
Fill holes act as a function that fill an image with white pixel. It fills all the holes that present in the image 
thus make the image full in white pixels. 
2.7. Creating Structuring Elements 
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This process using structural element in order to creates a large shapes of binary image from the 
pupil region. Since the system using a night vision camera, the region that shows the darkest color 1s only the 
pupil region. Thus the use of structuring element together with dilation can help to creates image with more 
prominent and larger. Structuring element comes with many parameters such as diamond, rectangles, octagon 
and disk structuring elements. All of the parameters come with specific setting for the structural elements. 
For this project, the suitable parameter is disk structuring element. Disk structuring elements will creates a 
disk-shaped of binary image with desired radius. But the desired radius must satisfy the non-negative integer. 
The larger the value of radius, the structuring elements will become larger. 


2.8. Pupil Localization using Region Props and Circular Hough 

When the pupil portion has been detected, the next step is to track this portion from the rest of the 
image. It can be obtained by detecting the comparison of the black and white of the image. To identify the 
portion of white pixels, the system for Algorithm 1 use regionprops function to creates properties that can 
measures and highlight the image region with bounding box, while algorithm 2 will use Circular Hough 
Transform to detect the round image and centralized the image. The Image will be highlighted with circle 
outline that actually the properties of Circular Hough Transform. 


2.9. Gaze Tracking Direction 

The input image given to the MATLAB for processing will produce an output image with the 
coordinates of the pupil. The output image will be a fixed size and the size will be divided into six blocks and 
the pupil coordinates in the center will indicate it as straight. Where the block in the second row and third 
column will indicate the position of the pupil as left and similarly the block in the second row and first 
column will indicate the pupil position as Right. [1] By setting the coordinates of each block. The output 
images will be considered as a rectangle with certain length and of breadth. 

But a modification of this method would be proposed instead of using the coordinates for image 
size, the proposed method would be identifying the direction of the gazing by calculating the angle of sight. 
The distance from user eye can be calculated from the webcam and the distance of the user eye can also be 
calculated. With these two values, the value of 8 can be determined. This can be understood by looking at 
the Theorem Pythagoras concept. But, the webcam itself has been provided with the viewing angle. The 
viewing angle take place with 60 degree total as we take an example of 320*240 output images, the 320 is 
the x-axis of the frames and the viewing angle of the webcam will be synchronize with it. The condition 
would be like this:- 

a. Block (2, 1) & (2, 2): The output coordinates of the iris satisfying below condition will detect the pupil 
position as Right.Condition: if ((Centroid >=Block (2, 2) & (2, 1))) 

b. Block (2, 3) & (2, 4): The output coordinates of the iris satisfying below condition will detect the pupil 
position as Straight.Condition: if ((Centroid > Block (2, 3) and centroid < Block (2, 2) && centroid > 
Block (2, 4) and centroid < Block (2, 5)) 

c. Block (2, 5) & (2, 6): The output coordinates of the iris satisfying below condition will detect the pupil 
position as Left. Condition: 1f ((centroids >=Block (2, 5) & (2, 6))) 





Figure 5. Direction detection method 
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3. RESULT AND DISCUSSION 

The first step of the localization of the pupil 1s by capture the image of the human eye in RGB color 
for the image acquisition process. The image format will be JPG format that can be read by MATLAB 
command. The image also must be stored in MATLAB directory so that the command window can read the 
image. 
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Figure 6: Normal RGB image into night vision image 


Figure 6 shows the image of an eye in original RGB color captured from night vision webcam. The 
night vision camera allows the infra-red light to immerse to it. Therefore, the pupil region can be clearly 
identified. 
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3.1. Algorithm Performance 
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Based on the Figure 9, it can be shown that the blue line process the image faster compared to the 
red plotted line. This is due to the function used by the blue line is taking less time to process. 
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Figure 9. Performance of the function used in Matlab 


Table 1. Time Performance for Algorithm 1 


Line Number Code Calls Total Time % Time 
12 data=getsnapshot (vid); 299 13.964 s 45.05% 
23 imshow (data); 299 9.4195 30.3% 
1 vid=videoinput(‘winvid.. 1 1.941 s 6.3% 
18 e=edge(BW6, canny’); 299 1.331 s 4.3% 
20 BW=imfill(e, holes’); 299 0.743 s 2.4% 
All other Lines 3.645 s 11.7% 
Totals 31.043 s 100% 
Table 2. Time Performance for Algorithm 2 
Line Number Code Calls Total Time % Time 
29 [centersBright, radiiBrigh... 299 11.819 s 30.3% 
11 data=getsnapshot (vid); 299 9.713 s 24.9% 
24 imshow (data); 299 9.354 s 24.0% 
1 vid=videoinput (‘winvid.. 1 1.867 s 4.8% 
17 e=edge(BWS,’ canny’); 299 1.200 s 3.1% 
All other Lines 5.008 s 12.9% 
Totals 38.953 s 100% 


3.2. Accuracy of Gaze Detection 

The last is the test for accuracy, Table 3, Table 4 and Table 5 shows the accuracy of gaze detection 
by moving the eyeball to the left and right. Both technique produce accurate gaze and the detection can be 
clearly seen. But the problem occurs for both techniques when the eye gazes to the most right which it 
indicates the angle of 36 degree. This is due to the surrounding light that immerse towards the eyes. 

For this project, the requirement is for fast computational with high accuracy. For both algorithms, it 
uses canny edge detection which detects the edge boundary of pupil region. But for time computational, 
Algorithm 1 process faster compared to Algorithm 2 with time different of 7.91 seconds. Therefore for 
robotics application that requires fast computation, Algorithm | is more suitable and can be implemented into 
a robotics wheelchair system. 


Table 3. Accuracy of Pupil for Center Gaze 
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Parameters Canny with Region Props Hough Circle 


Centroid at x-axis 


83 


Center=12 degree 





Table 4. Accuracy of Pupil for Left Gaze 


Parameters Canny with Region Props Hough Circle 


Centroid at x-axis 109 


Left = -24 degree 





Centroid at x-axis 136 





Left= -36 
degree 
Table 5: Accuracy of Pupil for Right Gaze 
Parameters Canny with Region Props Hough Circle 


ntroi -axi 
Centroid at x-axis AA 


Right = 24 degree 





Centroid at x-axis 136 


Right= 36 
degree 





4. CONCLUSION 

This paper has presented the development and analysis of an algorithm that can detect the pupil 
region by using regionprops and Circular Hough. The pupil region is then formulated to give a reference 
angle that can be used by robotic wheelchair. Detection of pupil region is done by using a night vision 
camera that is modified from a normal webcam. The use of night vision camera can help to maximize the 
edge boundary region since it can differentiate between iris and pupil. The webcam is attached with 4 IR 
LED that can give extra illumination towards the eyes so that the pupil will be more prominent. This is an 
advantage for the gaze detection because the pupil can still be detected even in a dark room. More important, 
The IR LED does not make the patient or user feels uncomfortable since the IR LED can only be detected by 
night vision camera. For future improvement, the webcam can be replaced with a camera that can zoom out 
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from wide distance. The current camera is placed in front of the user and this might make the user feel 
uncomfortable. 
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